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ABSTRACT 

A aodel for evaluating educational products is 
presented which is based on a simple philosophy: decide in advance 
what the final product ought to look likes then use this conception 
to prescribe methods for developing and neasuring it. In the 
preordinate aodel, five separate activities are identified which 
occur in approximately the following order: (1) selection of the 
product, (2) selection of the critical properties of the product, (3) 
making critical properties as operational as possible, (4) using 
critical properties to describe materials and developmental 
procedures or instruction, and (5) using critical properties to 
prescribe evaluation* Assumptions underlying the preordinate model 
are uncovered, and strengths and weaknesses of the aodel are 
reviewed, several steps for enriching the aodel are then prescribed. 
(RC) 
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Preordinate Models of Product Development: 
Implications for Alternative Approaches to Evaluation 
(Or, How to End Up Undernourished) 

D. Cecil Clark 
University of Washington 

The model I should like to describe and talk about is certainly 
not new. At least in a general form it can be observed throughout 
the history of American education with formal articulation as early 
as Tyler's work in the thirties. Stripped of its eruditic fluffiness, 
the model—or at least the philosophy behind it — is nicely simple: 
Decide in advance what the final product ought to look like then use 
this conception to prescribe methods for developing and measuring it. 
Although our thinking about the model has become more sophisticated and 
refined, the basic orientation remains: Define in advance, then 
prescribe. 

What about the word "Product" used in the title? Let us consider 
it in the broadest possible sense in order to apply the model most 
widely. A "product" can be some desired behavior of a student, an 
altered attitude on the part of the teacher, an artistic creation of 
a second grader, a functional team teaching arrangement, more effective 
communication among administrators, or any other desired outcome. 

I have chosen to describe the model operationally as it seems to 
appear on the educational scene. Here, it is messy and inelegant but 
faithfully realistic and repeatable. Here also we can most vividly 
observe its assumptions, strengths, and weaknesses. 

Description of the Model 

In the preordinate model, about five separate activities can be 
identified which occur in approximately the following order: 

1. Selection of the product. Almost without exception, 

educators using this model decide upon the product to be de- 
veloped as an initial activity. A product within the school 
setting is selected *n the basis of school philosophy, the 
board's recommendation, teacher interest, assessed needs or 



more typically, a combination of these. Unfortunately, the 
evaluator is seldom invited to participate in this initial 
activity. Decisions about what product will be selected, 
along with its worthiness, are made almost entirely by 
administrators and teachers within the system. To illustrate 
the model as it is described, suppose that persons in i. 
certain school district, acting upon the results of a needs 
assessment survey, select the product "Open concept classroom" 
as part of an individualizing effort at the elementary level. 

Selecting the critical properties of the product. This 
is in reality a defining activity in which the product is 
brought into existence by a description of its critical 
properties or attributes. These allow us to differentiate 
among products and also to determine when a product has in 
fact been developed. Here the teachers begin to ask, "What 
are the defining properties of this product?" Or, in other 
terms, ft What are the characteristics of an open classroom?" 
Obviously, a product can be assigned different sets of crit- 
ical properties by different groups of people. Also--and 
this is important-- the critical properties of a product can 
and do change; over time, over age groups, across socio- 
economic levels, and so forth. 

- Using our illustration, an "Open concept classroom" might 
be one in which! 

a. The students feel free. 

b. Each student can select and study any topic of his 
choice. 

c. The amount of time the teacher addresses the class 
as a unit is less than five percent. 

d. Visitors or non class members can come and go without 
disrupting what individuals within the class are doing. 

e. There are no permanent partitions. 

In times past, educationists have called in the evaluator 
to assist in the selection of critical properties. More 
recently, however, they have felt comfortable conducting this 
task by themselves. 



Making critical properties as operational as possible. 
In ttiany cases, critical properties are selected which are 
operational or, stated differently, which are measurable. 
Indeed, the major criterion for selecting a critical property 
has unfortunately been its measurabiiity. But this need not 
nor should be the case. Important critical properties can 
and ought to be selected regardless of their measurabiiity. 
For this reason activity two (selecting critical properties) 
has been separated from activity three (making them as oper- 
ational as possible). 

Typically, it is at this stage that the evaluator enters 
the picture. Sitting down with the educationist he recascr a 
critical property here, makes modifications there, adds one, 
omits another. But, he is not allowed considerable freedom 
in this endeavor. Rework to make more measurable is his 
theme. Value judgments about the worthiness of this or that 
critical property are generally thought to be outside his 
role as evaluator. 

Following through with our example, the evaluator might 
look at the critical property "The students feel free" and 
decide to sharpen it to "There is no restriction on physical 
movement or verbalizations of any student within the class- 
room. Likewise, there is no restriction on who enters or 
leaves the room. 9 * 

Using critical properties to prescribe materials and 
developmental procedures or instruction. " When the product 
is a learned behavior on the part of the student, the education- 
ist might ask, "What do these critical properties suggest in 
the way of learning experiences, materials to be selected, 
teaching strategies, and sequencing? 11 Hie more specific the 
critical properties, the greater their prescriptive power, 
and the narrower the range of potential materials and ex- 
periences. Since educationists have been traditionally strong 
in planning the "curriculum 91 they, more than anyone else, 
engage in this activity. 

In our example, the critical properties suggest a variety 



of development activities: Construct a room in which all 
partitions or screens are movable; develop a wide variety of 
resources which can be used by individual students and which 
will not compete with each other; help teachers develop 
instruction so the^ will not have to meet with the class as 
an entire unit more than five percent of the time; encourage 
outsiders to spend time in the room and make adjustments to 
reduce their interference. 

Using critical properties to prescribe evaluation. 
While the educationist is busily generating curricula or 
development procedures, the evaluator has been assigned the 
task of constructing an evaluation program. He, too, studies 
the critical properties but asks a different type of question: 
"How can I most directly and validly measure the attainment 
of this product?" Or, more precisely, "How can I measure 
attainment of the critical property <* which we are using as 
evidence for the product?" Unfortunu *iy, the evaluator is 
too often given sole responsibility fc. t'lis activity. 

Initially, he searches for aire ;'y developed instruments 
which can provide valid measurements of the critical pro- 
perties. More specifically, he looks for those that call forth 
the same characteristics identified in the critical properties. 
But, because critical properties are usually very specific, 
currently available instruments are found to be overly in- 
clusive in their measurements. The remaining option is to 
construct instruments custom tailored to these particular 
critical properties. Indeed, as the properties become snore and 
more specific, they become the evaluation instrument. 

Of all psychometric properties then, content (logical) 
validity becomes paramount in the development of evaluation 
within a preordinate model. If the product is a general skill 
on the part of the student, the conscientious evaluator will 
construct some sort of general evaluation plan and table of 
specifications in order to build validity into his measurements. 
The former device aids in obtaining instruments which call 
forth essentially the same behaviors as those identified by the 
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critical properties; the latter insures adequate sampling of 
these behaviors and the topics at which they are directed. 

How might the critical properties in our example prescribe 
an evaluation program? Recall that the first critical property 
was sharpened into the following: "There is no restriction 
on physical movement or verbalizations of any student within 
the classroom. Likewise, there is no restriction on who 
enters or leaves the room. 11 What is a direct and valid 
measurement of this critical property? An observational 
instrument such as a rating scale could be used to identify 
the behavior "degree of restriction. 11 A sample of randomly 
selected students could be unobtrusively observed over a 
period of time. Additionally, we might ask each student to 
respond on paper or in an interview to questions about his 
degree of restriction. Further, a tape recorder would capture 
the verbalizations of these students and after constructing a 
judgment scale, utterances could be analysed in tenrs of 
degree of restrictiveness. 

Similar kinds of procedures could be followed in develop* 
ing evaluation instruments for the other critical properties. 
The important point here is that when criticial properties 
are specific and operational, development of measurements 
with high content validity is rather straightforward and not 
overly difficult. 

Finally, after gathering and processing his data, the 
evaluator prematurely ends his involvement by turning over 
results to the educationist who is now left with the crucial 
task of interpretating and reconsoending. 

Assumptions Underlying the Model 

In order to think more exhaustively about the strengths and weak- 
nesses of the preordinate model, it should be helpful to uncover a few 
assumptions made by those who use this model : 

1. High quality critical properties can be identified prior to 

the product 1 s development. 

2. The critical properties of a product remain relatively constant 

over a period of time and over a variety of contexts. 

ERLC 
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3. The critical properties chosen are representative of all criti- 

cal properties that could have been selected. 

4. The measurahility of a property is an important criterion for 

selecting it as critical. 

5. Content validity is the most important psychometric characteristic 

of a set of measures. 

6. Evaluation should be limited to those critical properties that 

have been predetermined. 

Strengths of the Model 

1. The model is simple, internally consistent and enjoys vide 

applicability. 

2. The model is heavily presciptive. Once the critical properties 

of a product have been clearly identified, development and 
evaluation are effectively facilitated. 

3. The model stresses isomorphism among its components. At its 

most reductive level there is virtually a one-to-one 
correspondence between critical properties and evaluation. 
Suppose, for example, the reductive critical property 
"being able to add 2 plus 2 and arrive at 4." Evaluation 
consists of nothing more than asking the student to add 2 
plus 2. Although reducing critical properties to such an 
extreme is usally undesirable, the notion of tying evaluation 
* closely to critical properties is basically a sound one. 
Stated in more familiar language, we still want an instrument 
to measure what it is supposed to measure. 

Weaknesses of t he Model 

1. The model, in restricting evaluation, trades off richness for 

precision. 

2. The model- -or those of us who have used it~-lacks the sophisti- 

cation demanded by the complex process of educability. The 
model is at its best with simple products developed through 
simple means. The overall process of educability involves 
too many changes, too much subtlety, too much learner 
variability, too much incidental learning to be completely 
accomodated by a set of such simplistic activities. 
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3. Since critical properties prescribe evaluation, the quality of 

evaluation can be no higher than the quality of the critical 
properties, the latter of which is typically low. 

4. Neither educationist nor evaluator is presently skillful enough 

to predetermine an optimal and complete set of critical 
properties for a product. 

5. The model is unnecessarily rigid in limiting itself to only 

those critical properties which have been predetermined. 

Enriching the Model 

1. The preordinate nat of the model ought not to be lost but 

there is a need become more tentative 9 less rigid. 
Preidentif ied critical properties should be honest proto- 
types of later refinements. 

2. The prescriptive pouer of the model ought not be lost but it 

needs to be counterbalanced by a legitimate feedback 
function. Just as established critical properties prescribe 
evaluation, evaluation should turn around and provide new 
critical properties, or at least modifications of existing 
ones. 

3. Critical properties ought not be over defined. They should be 

left at a heuristic level; one that will evoke other critical 
properties thus providing a more complete definition of the 
* product. 

4. Evaluation ought to be more complete, varied, and exhaustive. 

Not only should we gather data on predefined critical pro- 
perties but on any aspect of the product or its development 
which might be helpful in making later decisions. Such an 
orientation ushers the evaluator into a more versatile role... 
he progresses from a scientist into a craftsman. A crafts* 
man is no less scientific. He is, rather, more skillful in 
applying the tools of his trade to a wider range of problems. 
And, if his bag fails to contain an appropriate tool, he 
possesses the ability and good sense to create a new one. 
Likewise, he is unconsnonly flexible in his trade. He can 
observe both carefully and crudely. He can gather data both 
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systematical ly and unsystematically ; he can be intentional 
or anecdotal; he uses experimental designs as well as de- 
scriptive surveys; he can call on multivariate analysis 
as well as frequency counts. His earmark as a true crafts- 
man, then, comes in his ability to gather the richest data 
for the problem at hand. 



